Field of the Invention
The present invention relates to image stabilization of captured image data.
Description of the Related Art
As the number of pixels and as the zoom ratio of image capturing apparatuses, such as digital cameras, increase, blurring of a captured image due to camera shaking poses a problem, and image capturing apparatuses with an image stabilization function have been widely used.
The image stabilization function normally is performed by a system which optically compensates for camera shaking by using a gyro sensor that detects an angular velocity caused by camera shaking and a driving device that controls the relative position between a lens and an image capturing device to compensate for camera shaking (to be referred to as an optical image stabilization system hereinafter).
Also, another system has been proposed (for example, Japanese Patent Laid-Open No. 2006-074693 (reference 1)). In this system, a plurality of images are continuously captured each for an exposure time free from the influence of camera shaking, that is, at a high shutter speed, and are synthesized by registering their positions, so as to generate an image free from the influence of camera shaking (to be referred to as a digital image stabilization system hereinafter). The digital image stabilization system does not require any gyro sensor, and can attain size and manufacturing cost reductions of an image capturing apparatus.
Furthermore, a technique which compensates for a blur by randomly opening and closing a shutter during an exposure period (to be referred to as a fluttered shutter hereinafter) and by deconvolution operations using information indicating opening and closing timings of the shutter (to be referred to as an fluttered shutter pattern hereinafter) has been proposed (for example, U.S. Patent Application publication No. 2007/0258706 (reference 2)). Since this technique called “coded exposure” corrects a blur by arithmetic operations, it does not require any registration of a plurality of images, as required in the technique of reference 1, thus decreasing the operation cost.
However, the technique of reference 1 assumes that the plurality of images do not include any blur. When a blurred image is included, the positions of the plurality of images often fail to be registered. For example, an imaging operation of a night scene requires an exposure time longer than or equal to a predetermined time so as to suppress noise in a captured image. However, when the exposure time is prolonged, each captured image is more likely to include a blur, and the positions of the plurality of images are more likely to fail to be registered.
The technique of reference 2 can correct image blur due to a blur generated in a certain direction, but it cannot correct image blur due to complicated blurs like movement of a person (to be referred to as a motion blur hereinafter). In other words, this technique can correct a blur caused by a uniform blur (including a motion blur of an object) for an entire image, but it cannot correct image blur caused by blurs including locally different blurs.